When compiling libostree, OSTREE_ENABLE_EXPERIMENTAL_API is managed via
config.h. However, g-ir-scanner doesn't use that and won't include the
experimental APIs unless the macro is defined through another means.
Without this, none of the experimental APIs were being included in the
gir data.

Both of these were inadvertently being included in the GIR. The
OstreeCmdPrivateVTable declaration ended up causing a g-ir-compiler
error because I wanted to add a function that had a return type that was
marked as skipped (OstreeRepoAutoLock). The ostree_cmd__private__
documentation just had the wrong name.

Currently ostree has no method of guarding against concurrent pruning.
When there are multiple repo writers, it's possible to have a pull or
commit race against a prune and end up with missing objects.

This adds an file based repo locking mechanism. The intention is to take
a shared lock when writing objects and an exclusive lock when deleting
them. In order to make use of the locking throughout the library in a
fine grained fashion, the lock acts recursively with a stack of lock
states. If the lock becomes exclusive, it will stay in that state until
the stack is unwound past the initial exclusive push. The file locking
is similar to GLnxLockFile in that it uses open file descriptor locks
but falls back to flock when needed.

The lock also attempts to be thread safe by storing the lock state in
thread local storage with GPrivate. This means that each thread will
have an independent lock for each repository it opens. There are some
drawbacks to that, but it seemed impossible to manage the lock state
coherently in the face of multithreaded access.

The API is a push/pop interface in accordance with the recursive nature
of the locking. The push interface uses an enum that's translated to
LOCK_SH or LOCK_EX as needed. Both interfaces use an internal timeout
field to decide whether to manage the lock in a blocking or non-blocking
fashion. The intention is to allow ostree applications as well as
administrators to control this timeout. For now, the default is a 30
second timeout.

Note that the timeout is handled synchronously in thread since the lock
is maintained in thread local storage. I.e., the thread that acquires
the lock needs to be the same thread that runs the operation. There may
be a way to offer an asynchronous version, but it's not clear exactly
how that would work since it would likely involve a separate thread that
invokes a callback when the locking operation completes.

https://bugzilla.gnome.org/show_bug.cgi?id=759442

Define an auto cleanup handler for use with repo locking. This is based
on the existing auto transaction cleanup. A wrapper for
ostree_repo_lock_push() is added with it. The intended usage is like so:

  g_autoptr(OstreeRepoAutoLock) lock = NULL;
  lock = ostree_repo_auto_lock_push (repo, lock_type, cancellable, error);
  if (!lock)
    return FALSE;

The functions and type are marked to be skipped by introspection since I
can't see them being usable from bindings.

Take a shared repo lock during a transaction to ensure that another
process doesn't delete objects.

Add exclusive repository locking to all the pruning entry points. This
ensures that objects and deltas will not be removed while another
process is writing to the repository.

Test that concurrent commits and prunes can succeed. Mostly this is a
check that the new locking works correctly and the concurrent processes
will properly wait until they've acquired the appropriate repository
lock.

Set the PYTHONUNBUFFERED environment variable during tests so that
python leaves stdout unbuffered. This is helpful when reading logs for
failures since the interleaved stdout and stderr will generally come out
in the right order. It's not perfect since tap-driver.sh does some
special redirection to the log file, but it's an improvement.

When checking out objects, take a shared lock to ensure objects aren't
deleted. If the repo isn't writable, the locking is a noop. This ensures
that the repository owner can make robust checkouts, but it still allows
other users to make a likely to be successful checkout if the repo mode
supports it. During garbage collection, take an exclusive lock while
deleting uncompressed objects.

Add a repository option, core.lock-timeout that controls the default
lock timeout used. This allows administrators to have some influence
over locking operations. For instance, critical repositories where
operations should not fail could be configured to block when managing
locks by setting lock-timeout to -1.

Exercise the ostree_repo_lock_push() and ostree_repo_lock_pop() APIs
directly via pygi. This is a better unit test than relying on concurrent
commit and prunes to check that the locking works correctly.

Ensure that the commit object doesn't get deleted while writing out the
detached metadata. This is primarily to guard against dangling
commitmeta files if racing with a prune since the commit object could
still be updated while this is happening.

This ensures that the commits and deltas don't get deleted while
generating the summary metadata.

Make sure that the summary file does not get replaced while generating
summary.sig by taking an exclusive lock. It would be bad to publish a
mismatching signature.

Make sure that neither commit nor any of the objects they reference are
deleted while generating a delta.

Allow the ostree commands to use ostree_repo_set_lock_timeout() even
when ostree is built without experimental API. When it's no longer
experimental, this should be removed.

Builtins that require repository locking can add options with the
OSTREE_BUILTIN_FLAG_LOCKING flag set to add a --lock-timeout option.
This allows the user to control how long to wait to acquire the
repository lock.

The cmdprivate version of ostree_repo_set_lock_timeout() is used until
it's no longer experimental.

All of these commands call one of the libostree functions that take
locks, so add the --lock-timeout option to them.

Allow the ostree commands to use ostree_repo_auto_lock_push() and
ostree_repo_auto_lock_cleanup() even when ostree is built without
experimental API. When it's no longer experimental, this should be
removed.

Unfortunately, this requires including ostree-repo-private.h from
ostree-cmdprivate.h. That includes otutil.h, which the systemd generator
build wasn't prepared to handle.

Since ostree_repo_auto_lock_cleanup is not directly available, a
separate type, wrappers and autoptr declaration are needed.

Although the libostree functions will take locks as needed, its best to
ensure fsck has exclusive access to the repo for checking consistency.

Admin builtins that require repository locking set the
OSTREE_BUILTIN_FLAG_LOCKING flag to add a --lock-timeout option. This
needs to be used in conjunction with OSTREE_BUILTIN_FLAG_NO_REPO since
the the timeout can only be applied once the repo is gathered from the
sysroot. This allows the user to control how long to wait to acquire the
repository lock.

The cmdprivate version of ostree_repo_set_lock_timeout() is used until
it's no longer experimental.

All of these commands call one of the libostree functions that take
locks, so add the --lock-timeout option to them.

Make sure the number of committers is even since we only create half as
many trees.